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ABSTRACT 

The Robot assembly comprises of one robot helicopter and two unmanned ground vehicles, they are AT 
Robots. The AVATAI (Aerial Vehicle Autonomous Tracking andlnspection) is based upon the Bergen 
Industrial RC model helicopter. The following sensors/electronics are onboard all robots; GPS, color 
CCD camera, wireless video and embedded PC/104 boards running QNX (a realtime OS). Wireless 
Ethernet provides a 2.0 Mbps multiway communication link between all robots and the operator. The 
AVATAI also has an Inertia! Navigation System and the Pioneers have seven ultrasound sensors for 
obstacle detection. The helicopter is controlled partly by the onboard control system and in part by a 
human pilot. The Pioneers are fully autonomous. To date we have demonstrated 1 ) High-level tasking 
by a human operator of the robot group to cooperatively patrol a designated area and 2) Automatic 
retasking of the robots by simulated "alarms" on the area perimeter. The robots navigate to the alarm 
scene to provide live video while maintaining basic patrolling functionality. 

KEY WORDS: GPS, AVATAI, Pioneer, OCU, QNX, Navigation System 



I. INTRODUCTION 

Our experimental test bed (Figure 1) is composed of a robot helicopter and two mobile ground robots 
thus making it a morphologically heterogeneous group. The advantages of using a group of robots to execute 
coordinated activity have been discussed extensively in the literature. However heterogeneous groups typically 
involve higher overhead in terms of system maintenance and design. The USC Robotics Research Laboratory 
designed a heterogeneous group consists of a robot helicopter and several mobile robots for inspection and 
observation applications. 




Figure 1: The experimental testbed consisting of the AVATAI (Autonomous Vehicle Aerial Tracking and 
Inspection) robot helicopter and two Pioneer AT mobile robots. 
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In the work described here, we focus on an experimental task motivated by an application where a 
single person has control and tasks the robot group at a high level. A possible application is alarmed with 
security in urban areas where a single guard would need to control and monitor several (possibly heterogeneous) 
robots. In the application described here the robot group patrols an outdoor open area (a few obstacles are 
present on the ground) defined by a perimeter. The boundary is defined by a set of vertices (in GPS coordinates) 
which when connected by line segments form the boundary of a closed convex irregular polygon. In the work 
reported here the robot helicopter is flown by a human pilot for purposes of safety and speed. However we rely 
on a human pilot to transition from a stable hover at one point in space to another. Both ground robots are fully 
autonomous in the experiments reported here. 

The robots in the experiments cover the area bounded by the perimeter and send imagery back to a 
human operator via a wireless video downlink. The operator is able to set high level goals for the ground robots. 
An example of a high level goal is "follow the helicopter." If a particular ground robot is given this high level 
goal it will stop patrolling and will follow the helicopter as it moves. The motivation behind this is to allow the 
ground robots to "take a closer look" at areas which the operator finds interesting based on aerial imagery. The 
operator can designate points of interest within the perimeter which serve as area markers for robots to 
periodically explore. The basic idea behind the implementation of the control programs is however to achieve 
robust functionality without explicit top-down planning. Rather the overall behavior of the group is the result of 
interacting control systems that run on the individual robots which essentially allow each robot a high degree of 
autonomy. 

II. HARDWARE AND SOFTWARE DESCRIPTION 
2.1 AVATAI Hardware 

The current AVATAI, shown in Figure 1, is based upon the Bergen Industrial Helicopter, a radio 
controlled (RC) model helicopter. It has a two meter diameter main rotor, is powered by a 4.0 horsepower twin 
cylinder gas engine and has a payload capability of approximately 10 kilograms. The helicopter has five 
degrees of control: main rotor lateral and longitudinal cyclic pitch, tail rotorpitch, main rotor collective pitch and 
engine throttle. The first three control the roll, pitch and yaw of the helicopter while the last two control its 
thrust. The helicopter can be controlled by a humanpilot using a hand held transmitter which relays pilot control 
inputs to an onboard radio receiver using the 72 MHz frequency band. The receiver is connected to five 
actuators, one for each degree of control on the helicopter. For autonomous operation, these pilot control inputs 
are replaced by computer generated control inputs. A block diagram of the AVATAI system; including sensors, 
onboard and off board computing resources, wireless communication links and electrical power sources, is 
given in Figure 2. A variety of sensors are mounted on the AVATAI that provide information about the state of 
the helicopter as well as the environment in which it operates. An integrated Global Positioning System/Inertial 
Navigation System (GPS/INS) device, consisting of a GPS receiver and an Inertial Measurement Unit (IMU), is 
the primary sensor used for low-level control of the helicopter. The GPS/INS provides position (latitude, 
longitude and altitude), velocity (horizontal and vertical), attitude (roll and pitch), heading (yaw), delta theta and 
delta velocity information. This particular GPS receiver can only track 4 satellites at once and consequently 
provides a relatively poor estimate of current latitude and longitude as compared to other available receivers. 
So, a second standalone GPS receiver is used that can track up to 12 satellites at once. 

This improves the standard deviations of the estimates of latitude and longitude from 4.5 meters for the 
4-channel GPS unit down to 20 centimeters for the 12-channel unit. This GPS receiver is used for the high-level 
(guidance and navigation) control of the helicopter. A downward facing ultrasonic (sonar) transducer provides 
altitude information and a RPM sensor mounted on the main rotor mast measures engine speed. A downward 
looking color CCD camera provides visual information of the area below the AVATAI. 
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Figure 2: AVATAI system block diagram 

Onboard computing needs are met using a number of commercial off the shelf (COTS) PC/104 boards 
and one additional custom built PC/104 board. The main processor board contains a 486DX4 CPU which runs at 
133 MHz, has 16 Mbytes of RAM and 40 Mbytes of solid state disk on chip (DOC). The DOC contains both the 
realtime operating system (RTOS) and flight software. The 486DX4 boots up the RTOS at system power up, 
executes all flight control and image processing software and provides an interface to other PC/ 104 boards. 
These boards include a timer/counter board used both to generate actuator commands and read pilot commands, 
a custom built PC/ 104 board that allows switching between human generated and robot generated commands on 
an actuator by actuator basis as well as acting as an interface to the RPM sensor and sonar, a serial port board 
for interfacing to the GPS/INS and standalone GPS, a color video frame grabber for the CCD camera and a 
PC/104 to PCMCIA interface board to allow the use of PCMCIA cards onboard the AVATAI. A 2.4 GHz 
wireless Ethernet PCMCIA card provides a multiway 2.0 Mbps communication link between the AVATAI, 
other robots and a human using an operator control unit (OCU). (The OCU comprises the off board computing 
resources and will be described in detail later). The human receives grabbed video frame information and other 
telemetry from the AVATAI and sends high-level tasking commands to the AVATAI via this link. In addition, 
differential GPS corrections are sent from the OCU to the GPS receivers onboard the AVATAI through the 
wireless Ethernet to improve the GPS performance. A live video feed, provided by a one way 1.3 GHz wireless 
video link from the CCD camera, is displayed on a monitor. Nickel-metal hydride (NiMH) and lithium-ion (Li- 
Ion) batteries supply power to the electronics. The GPS/INS is relatively power hungry, requiring 0.8 amps at 
+24 volts and so a dedicated 12-volt, 3.5 amp-hour NiMH battery is connected to a DC/DC converter to produce 
+24 volt power. The electronics can operate for roughly an hour with this battery supply. Mission length is 
limited by the flight time of the helicopter on a single tank of gasoline, which is approximately 30 minutes in 
duration. 

2.2 AVATAI Software 

The operating system used is QNX; a UNIX-like, hard real time, multitasking, extensible POSIX 
RTOS with a small, robust microkernel ideal for embedded systems. The flight software encompasses all 
remaining software running onboard the AVATAI. This currently includes (or is planned to include): 

1 . low-level software drivers for interfacing with sensors and actuators 

2. Flight control software for guidance, navigation, and control. 

3. learning software 

4. self-test software including built in test (BIT) and health checks software for increasing robot fault 
tolerance 

5. vision processing software running on the frame grabber 
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Figure 3: Pioneer system block diagram 



All software is written primarily in C/C++, with assembly used only when required for speed of 
execution. Custom drivers have been written for the timer/counter card, the GPS/INS and GPS units, the frame 
grabber and the wireless Ethernet PCMCIA card. The remaining software is still under development. 

2.3 Pioneer Hardware 

The Pioneer AT robots used in this work is identical to each other. Each robot is four-wheeled base 
with skid steering. The wheels on the left are coupled mechanically as are the wheels on the right resulting in 
two degrees of freedom in the drive. Turning is accomplished by a speed differential between the left and right 
sides. Each robot has a Lithium-Ion (Li-Ion) battery pack (Two 10.8 volt, 4.05 amp-hour Li-Ion) for the 
electronics. The motors are powered by a lead acid battery that allows up to 4 hours of operation on hard 
surfaces and approximately one hour on grass. Each robot is equipped with a ring of seven front looking sonars 
which are controlled by a low-level Motorola 6811 microcontroller. The wheel speeds are available through 
encoders. The low-level 681 1 board is connected to a PC/104 stack via a serial connection. The Pioneer also has 
a vision system comprised of a camera on a pan-tilt head controlled by a Motorola 68332 board running the 
Congnachrome color vision system. The main PC/104 processor board contains a 486DX4 133MHz CPU, 4 
Mbytes of RAM and 40 Mbytes of solid state disk on chip which contains both the real time operating system 
(RTOS) and control software. Each robot is equipped with a NovaTel GPS system connected to the PC/104 
processor via a serial port. Additional sensors include a compass and a gyroscope. The gyroscope is connected 
to a A/D card on the PC/104 stack. A 2.4 GHz wireless Ethernet PCMCIA card provides a multiway 2.0 Mbps 
communication link between each Pioneer and the other robots. A live video feed, provided by a one-way 2.3 
GHz wireless video link from the CCD camera, is displayed on a monitor. A block diagram of the Pioneer 
hardware is given in Figure 3. 

2.4 Pioneer Software 

The control system for the Pioneer AT (described in the next section) is all written in C and runs under 
QNX on the PC/104 stack described above. The software includes 

1 . Low-level software drivers for interfacing with sensors specifically software for the wireless Ethernet driver 
and the GPS driver. 

2. control software for obstacle detection and avoidance, navigation and mapping 

2.5 OCU Hardware 

The operator control unit is implemented using a Toshiba Tecra 510CDT laptop PC based upon a 133 
MHz Pentium CPU. It has 144 Mbytes of RAM, a 4.3 Gbyte hard drive, a 12.1 inch high -res color monitor, a 
CD-ROM and an Ethernet connection. The QNX operating system is installed as well as the Watcom C/C++ 
compiler. A docking station expands the capabilities of the laptop by providing two full-length expansion slots 
for standard ISA and 32-bit PC Expansion Cards and onehalf-length 32-bit PC expansion slot. It also has a 
selectable bay for installing additional hardware such as CD-ROM drives or floppy drives. A 1.44M byte 
floppy drive has been installed in the slot.The 510CDT has multiple functions and is used during all phases of 
the project; including development, integration and test. The primary purpose of the laptop is to function as a 
wireless OCU for communication and tasking of the robots. A 2.4 GHz wireless Ethernet device is connected to 
the Toshiba, providing a multiway connection between the human at the OCU and the wireless PCMCIA cards 
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onboard the robots. Additional functions of the 510CDT include the following: First, it provides a software 
development environment through the use of the QNX operating system and the Watcom C/C++ Compiler. 
Using this environment code is developed and tested. Also, using RCS, a QNX software version control utility, 
software configuration management is implemented. Second, the 4.3 Gbyte hard drivesprovide long-term 
storage capability for mission data. Third, the docking station provides an ISA slot for a GPS receiver used to 
produce differential GPS corrections for use by the mobile robots. 

III. USER INTERFACE 

The user interface for the system is implemented under QNX using the Phab GUI development system. 
The basic layout of the interface is deliberately kept simple so as to allow an inexperienced operator to quickly 
learn how to use it. A screenshot of the interface is shown in Figure 4. The user interface allows the operator to 
examine telemetry from any robot, task individual robots to do specific activities (such as following, patrolling 
etc.) and monitor the location of the robots in a 2D plan view. In addition, TV monitors show the operator a live 
wireless video feed from each of the individual robots. Tasking is done by selecting a robot with the mouse and 
clicking on one of the tasks available in the form of the task list popup menu. 
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Figure 4: A screenshot of the user interface 



IV. RESULTS WITH TEST TASKS 

The system described here has been tested with two simple tasks to date. The operator initializes the 
system with a set of GPS coordinates that define the perimeter of interest. The robot locations are registered to a 
common coordinate frame which is displayed to the operator on the user interface. The first task used for testing 
was a cooperative patrolling activity by the robots. The helicopter was flown by the human pilot and the Pioneer 
ground vehicles followed it on the ground using GPS. The only input from the operator is at the start of the task. 
When the helicopter is fully autonomous we expect to be able to load a flight plan into it (in the form of several 
via points) and have the rest of the robots follow in loose formation. 

The second task we have used for testing involves the automatic retasking of the robots by simulated 
"alarms" on the area perimeter. The robots navigate to the alarm scene to provide live video while maintaining 
basic patrolling functionality. When an alarm goes off (alarms are simulated by user keyboard input to the 
system), the robot nearest to the alarm disengages the patrolling behavior and navigates to the site of the alarm. 
Since this is a reconnaissance style scenario the robot simply returns a close-up view of the scene of the alarm to 
the operator in the form of a live video feed. The other robot continues patrolling. A detailed analysis of the 
behavior of the group as a whole is the subject of a forthcoming paper [Sukhatme 99]. Videotapes of the system 
are available at http://robotics.usc.edu/brochure/afv.html. 
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